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Introduction 

The  purpose  of  this  research  was  to  compare  a  current  instructional  approach  used  in  one 
course,  C510:  Strategic,  Operational,  and  Joint  Environments,  of  the  Command  and  General 
Staff  Officer  Course  (CGSOC)  to  an  alternative  instructional  approach  derived  from  the  Problem- 
Based  Learning  (PBL)  approach.  Specifically,  this  research  sought  to  determine  if  (1)  a  cued 
retrieval  instructional  strategy  in  a  problem  oriented  instructional  situation  would  foster  encoding 
specificity,  hence,  retention,  and,  (2)  a  focus  on  identification  of  learning  issues  by  participants 
would  foster  an  orientation  to  self-directed,  continuing  learning. 

Problem-based  learning  is  an  instructional  innovation  in  medical  education  first  adopted  by 
McMaster  University  in  the  mid  1960s.  Since  that  time,  approximately  60  medical  schools  world¬ 
wide  have  implemented,  at  least  in  part,  a  problem-based  learning  approach  in  their  curricula 
(Norman  &  Schmidt,  1992).  Albanese  and  Mitchell  (1993)  described  PBL  as  “an  instructional 
method  characterized  by  the  use  of  patient  problems  as  a  context  for  students  to  learn  problem¬ 
solving  skills  and  acquire  knowledge  about  the  basic  and  clinical  sciences”  (p  53). 

Barrows  (1986)  articulated  four  major  educational  goals  of  PBL.  The  first  is  the 
structuring  of  knowledge  for  use  in  practice  contexts.  Imbedded  in  this  goal  is  the  notion  that 
retention,  recall,  and  application  of  new  knowledge  is  enhanced  as  knowledge  is  acquired  in  a 
problem  context.  A  second  goal  of  PBL  is  the  development  of  reasoning  processes  including 
hypothesis  generation,  inquiry,  data  analysis,  problem  synthesis,  and  decision-making.  The  third 
goal  is  the  development  of  self-directed  learning  skills  deemed  critical  for  practitioners  as  they 
confront  an  expanding  knowledge  base  and  unusual  or  unique  problems  in  practice.  The  final 
major  goal  of  PBL  is  to  enhance  intrinsic  motivation  for  learning.  The  rationale  for  this  goal  is 
that  the  opportunity  to  solve  problems  in  a  simulated  practice  situation  is  inherently  motivating. 
One  of  the  challenges  in  research  on  the  medical  school  PBL  model  is  its  multiple  and 
overlapping  theoretical  foundations  These  foundations  include  information  processing  theories, 
constructivist  perspectives  of  learning,  and  cooperative  learning.  Furthermore,  because  PBL  is 
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generally  implemented  in  a  cooperative  learning  environment,  it  purports  to  provide  learners  with 
the  opportunity  to  elaborate,  link,  interpret,  and  explain  new  knowledge  in  a  group  context. 

Background 

Research  to  date  on  PBL  has  focused  on  two  major  areas:  (1)  program  evaluation  issues, 
and,  (2)  comparisons  of  students’  achievement  on  medical  boards  and  clinical  performance 
assessments  in  conventional  vs.  problem-based  curricula.  Recently,  research  has  addressed  the 
underlying  learning  processes  of  PBL  that  produce  the  effects  that  have  been  reported  on 
achievement,  clinical  behavior,  use  of  informational  resources,  and  study  patterns  (Caplow, 
Donaldson,  Kardash,  &  Smith,  1994)  However,  no  research  to  date  has  investigated  any  of  the 
specific  underlying  theoretical  constructs  of  PBL 

One  area  of  potential  research  about  PBL  is  the  problem  solving  process  Bransford, 
Sherwood  &  Sturdevant  (1987)  have  articulated  five  steps  to  the  problem  solving  process:  (1) 
problem  identification;  (2)  problem  definition;  (3)  exploration  of  alternative  solutions,  (4)  acting 
on  a  solution;  and,  (5)  assessing  the  effects  of  the  solution.  Phye  (1992)  made  a  distinction 
between  academic  problem  solving,  which  is  based  on  prior  instruction,  and  creative  problem 
solving,  which  is  not  based  on  prior  instruction.  There  is  an  extensive  body  of  research  on  the 
various  cognitive  processes  and  related  instructional  strategies  involved  in  successful  academic 
problem  solving,  the  review  of  which  is  beyond  the  scope  of  this  report.  Briefly,  however, 
according  to  Bransford,  Sherwood  &  Sturdevant  (1987),  activation  of  prior  knowledge  is 
important  in  the  identification  and  definition  of  a  problem  The  activation  of  acquired  knowledge 
for  the  problem  situation  depends  on  how  or  if  it  was  stored  in  long  term  memory.  In  other 
words,  the  knowledge  must  be  both  available  and  accessible  for  application  to  the  problem 
situation. 

A  second  area  of  potential  research  is  on  the  transfer  of  acquired  knowledge  or 
competencies  to  an  actual  practice  situation.  Phye  (1992)  made  a  distinction  between  transfer 
appropriate  processing  and  transfer  appropriate  procedures.  In  transfer  appropriate  processing, 
the  focus  of  instructional  strategies  is  on  facilitating  encoding  specificity  for  the  transfer  process 
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by  anchoring  instruction  in  simulated  “real  life”  contexts.  An  example  of  this  is  the  instructional 
strategy  of  PBL  used  in  medical  schools  of  using  patient  cases  as  the  stimulus  to  acquire  new 
knowledge.  In  this  example,  the  patient  case  serves  as  a  context  for  encoding  specificity: 
providing  a  context  for  the  acquisition  of  knowledge  similar  to  the  actual  practice  situations  in 
which  they  are  applied.  The  activation  of  prior  knowledge  of  the  learner  that  is  relevant  to  the 
case  is  based  on  spontaneous  retrieval.  According  to  Tulving  (1983),  spontaneous  retrieval 
occurs  only  if  properties  of  the  new  situation  are  similar  to  properties  of  the  situation  in  which  the 
knowledge  was  acquired  (encoding  specificity  assumption).  Therefore,  at  issue  is  the  encoding 
specificity  of  knowledge  at  initial  acquisition  and  its  relationship  to  availability  and  accessibility 
(retrieval)  and  application  to  a  new,  but  similar  situation  (transfer).  In  transfer  appropriate 
procedures,  the  focus  of  instructional  strategies  is  on  facilitating  access  of  prior  knowledge, 
therefore,  memory  retrieval  rather  than  encoding  specificity  is  emphasized.  An  example  of  an 
instructional  strategy  that  would  facilitate  access  of  prior  knowledge  to  apply  to  a  problem 
situation  would  be  cued  retrieval  strategies,  such  as  questioning,  prior  to  the  transfer  task. 

Statement  of  the  Problem 

The  medical  school  PBL  model  is  a  comprehensive  curriculum  that  uses  several  broad 
instructional  approaches  (e  g.,  problem-oriented  learning,  cooperative  learning  environments).  A 
cornerstone  instructional  strategy  of  the  medical  school  PBL  approach  is  the  use  of  a  simulated 
situation  (a  patient  case)  to  facilitate  acquisition  and  retention  of  knowledge  for  transfer  to  actual 
practice  situations.  Analysis  of  one  of  the  broad  goals  of  PBL— structuring  knowledge  for  use  in  a 
practice  setting-  indicates  that  it  is  based  on  several  underlying  theoretical  constructs  derived 
from  cognitive  theories  of  learning.  These  constructs  are  anchored  instruction  (or  situated 
cognition),  spontaneous  retrieval,  encoding  specificity,  and  transfer  appropriate  strategies. 
However,  to  date,  research  on  the  medical  school  PBL  approach  has  not  addressed  these 
individual  constructs  and  their  relation  to  knowledge  acquisition  and  retention. 

Bridges  &  Hallinger  (1992,  1995)  suggest  an  alternative  to  the  medical  school  PBL 
model.  This  alternative  model  shares  the  goals  and  is  informed  by  many  of  the  same  cognitive 
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learning  theory  constructs  (i.e.  anchored  instruction,  encoding  specificity,  and  transfer  appropriate 
strategies)  as  the  medical  school  model.  But  it  differs  from  the  medical  school  model  in  one 
important  way;  the  learning  objectives  and  questioning  are  under  the  control  of  the  instructor 
rather  than  the  learners.  Objectives  for  the  problem  are  specified  in  advance  by  the  instructor,  and 
guiding  questions  are  provided  learners  to  direct  them  to  key  concepts,  assist  them  in  thinking 
through  the  problem,  and  stimulating  them  to  approach  the  problem  from  multiple  perspectives 
Therefore,  in  contrast  to  the  medical  school  model,  the  Bridges  and  Hallinger  model  employs 
cued,  rather  than  spontaneous,  retrieval  of  prior  knowledge,  as  well  as  strategies  that  direct 
learners  to  content  relevant  to  problem  resolution.  Yet,  research  on  this  alternative  PBL 
approach  has  also  failed  to  address  underlying  constructs  of  cognitive  learning  theory  and  their 
relation  to  knowledge  acquisition  and  retention. 

Still  other  strategies  exist  for  instructors’  use  in  facilitating  the  acquisition  and  retention  of 
knowledge  for  transfer  to  actual  practice  situations.  An  example  is  the  approach  used  in  the  C510 
(Strategic,  Operational,  and  Joint  Environments)  course  offered  at  Ft.  Leavenworth  as  part  of  the 
Command  and  General  Staff  Officer  Course  The  typical  format  for  this  course  is  one  in  which 
students  acquire  knowledge  in  more  conventional  ways,  such  as  reading,  lecture,  and  discussion, 
and  are  then  presented  with  a  problem  (transfer  task)  that  requires  the  application  of  acquired 
knowledge.  In  this  case,  anchored  instruction  at  the  time  of  knowledge  acquisition  is  not  used, 
hence  limiting  deliberate  attempts  via  an  instructional  strategy  to  facilitate  encoding  specificity.  In 
the  absence  of  specific  instructional  strategies  to  facilitate  retrieval  of  prior  knowledge, 
spontaneous  retrieval  is  used  by  the  learner.  However,  the  use  of  an  instructional  strategy  to 
stimulate  recall  when  the  problem  situation  is  presented  after  knowledge  acquisition  (through 
conventional  means,  such  as  reading,  lecture,  discussion)  provides  the  opportunity  for  encoding 
specificity  at  the  time  of  transfer.  A  major  speculation  of  this  study  was  that  the  use  of  a 
questioning  strategy  designed  to  stimulate  cued  retrieval  strategies  may  strengthen  the  students’ 
access  of  relevant  prior  knowledge  for  application  to  the  problem.  Therefore,  a  modified  PBL 
approach,  drawn  from  strategies  employed  in  the  medical  school  and  the  Bridges  and  Hallinger 
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models  of  PBL,  was  introduced  into  the  instructional  design  of  the  C5 10  course  for  the  purposes 
of  testing  this  speculation  about  cued  retrieval  and  access  of  relevant  prior  knowledge. 

An  orientation  towards  self-directed  learning  by  students  in  a  PBL  curriculum  has  also 
been  noted  in  the  literature  Specifically,  Vernon  and  Blake  (1993),  in  a  meta-analysis  of  research 
on  PBL,  reported  that  students  in  a  PBL  curriculum  engaged  in  more  self-directed  learning  (e  g., 
greater  use  of  the  library  and  use  of  self-selected  rather  than  faculty-selected  reading  materials)  In 
addition,  they  found  that  students  in  a  PBL  curriculum  placed  greater  emphasis  on  understanding 
material  rather  than  memorization  or  rote  learning  than  did  students  in  a  traditional  curriculum 
Blumeberg  and  Michael  (1992)  also  reported  that  PBL  fosters  self-directed  learning.  Their 
conclusion  was  based  on  student  self-reports,  accreditation  and  program  evaluation  documents, 
and  library  circulation  statistics.  Based  on  these  findings,  a  second  speculation  of  this  study  was 
that  the  implementation  of  a  specific  instructional  strategy  to  lead  students  to  the  identification  of 
learning  issues  (needed  areas  of  learning  to  solve  the  problem)  would  foster  an  orientation 
towards  self-directed  learning.  Therefore,  this  strategy  was  introduced  into  the  design  of  the 
C510  course  to  test  this  speculation  about  self-directed,  continuing  learning 

Method 

Procedure 

The  three  member  research  team  (Caplow,  Donaldson,  and  Preczewski)  first  met  with  the 
major  actors  relevant  to  C510.  These  included  the  Director  of  the  Command  and  General  Staff 
School,  the  Director  of  the  Department  of  Joint  and  Combined  Operations  (DJCO),  and  the 
course  directors  for  C510.  This  meeting  served  to  outline  the  proposed  research  methods  to  those 
responsible  for  the  conduct  of  the  course.  Approval  to  access  four  different  staff  groups  of  C5 10 
was  granted  and  faculty  volunteers  were  secured. 

The  research  team  next  spent  several  days  observing  the  faculty  development  seminars 
designed  to  prepare  DJCO  instructors  to  teach  C5 10.  The  course  is  comprised  of  eight  lessons 
with  the  first  six  providing  the  knowledge  necessary  to  successfully  complete  a  capstone 
application  task  in  Lesson  7.  This  task  requires  students  to  prepare  a  strategic  estimate  and 
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analysis  concerning  a  notional  crisis  emerging  between  North  and  South  Korea.  Lesson  8  is  a 
transition  lesson  for  a  subsequent  course. 

In  addition  to  attending  faculty  development  seminars,  the  team  conducted  a  detailed 
analysis  of  all  the  course  materials.  The  key  knowledge  and  concepts  related  to  the  task 
requirements  of  Lesson  7  were  determined  from  the  analysis  of  course  materials,  interviews  with 
the  course  directors,  and  observations  of  the  faculty  development  seminar.  A  key  concept  list  was 
developed  and  presented  to  the  course  director  for  a  final  check  on  completeness  and  accuracy 
These  key  concepts  became  one  of  the  dependent  variables  measured  upon  completion  of  the 
lesson. 

Four  C510  staff  groups  of  15-17  students  in  each  staff  group  participated  in  the  study. 
Two  of  these  groups  were  the  comparison  groups  and  two  were  the  study  groups.  The  same 
instructor  taught  both  comparison  groups  and  the  same  instructor  taught  both  study  groups,  and 
the  course  content  was  the  same  for  all  four  groups.  The  course  is  offered  on  a  rotating  basis. 
During  the  first  rotation  in  August,  one  comparison  group  and  one  study  group  were  observed 
during  Lessons  6,  7,  and  8  and  data  were  collected.  When  the  course  was  offered  on  the  second 
rotation  (September),  data  were  collected  from  one  comparison  and  one  study  group.  These  two 
groups  were  not  observed.  The  first  rotation  of  the  comparison  and  study  groups  were  observed 
to  identify  instructional  approaches  and  activities.  A  member  of  the  researcher  team  met  with  the 
instructors  prior  to  the  second  rotation  of  the  course  to  gain  assurance  that  instructional 
approaches  and  activities  would  remain  similar  to  those  used  during  the  first  rotation.  Upon 
completion  of  the  second  rotation  of  the  course,  the  instructors  were  interviewed  to  check 
whether  or  not  the  class  did  indeed  proceed  in  a  fashion  similar  to  the  first  observed  rotation. 

They  reported  no  significant  differences  in  the  classroom  proceedings  nor  the  quality  of  the  final 
group  product  between  the  staff  groups  in  the  first  rotation  and  those  in  the  second  rotation. 

The  instructor  of  the  comparison  groups  used  the  standard  instructional  approach,  as 
presented  in  the  faculty  development  seminar.  In  these  groups,  the  instructional  strategies  used  in 
Lessons  1  through  6  included  lecture,  reading,  and  group  discussion.  In  preparation  for  Lesson  7, 
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students  were  provided  with  an  overview  of  the  strategic  estimate  process  and  then  given  the  task 
scenario  (see  Appendix  A)  to  complete  as  a  total  group. 

The  instructional  strategies  used  by  the  instructor  of  the  study  groups  for  Lessons  1 
through  6  also  included  lecture,  reading,  and  group  discussion.  However,  prior  to  being 
presented  with  the  task,  these  groups  were  each  divided  into  two  groups  of  eight  and  given 
questions  designed  to  stimulate  cued  recall  of  relevant  knowledge  and  concepts  learned  in 
Lessons  1-6  and  to  be  applied  to  the  task  (see  Appendix  B).  Questions  were  developed  to 
stimulate  cued  recall  of  information  relevant  to  two  parts  of  the  scenario:  the  Background  and  the 
Situation.  After  the  two  groups  of  eight  responded  to  the  questions  relevant  to  the  Background  of 
the  scenario,  they  re-grouped  as  a  total  class  and  provided  a  briefing  to  the  instructor  Then,  they 
split  into  the  two  groups  of  7-8  students  to  address  the  questions  relevant  to  the  Situation  and, 
again,  re-grouped  for  a  briefing.  This  allowed  the  instructor  to  check  and  remediate  learning  prior 
to  the  students  completing  the  strategic  estimate  task  which  culminated  in  a  final  briefing. 

Data  Collection 

Upon  completion  of  the  briefing  of  the  scenario  task  (Strategic  Estimate)  of  Lesson  7, 
students  were  immediately  asked  to  remain  in  the  classroom  to  participate,  on  a  voluntary  basis,  in 
data  collection  relevant  to  their  learning.  All  of  the  students  remained,  but  as  reported  in  the 
results,  not  all  participated  in  all  components  of  the  data  collection. 

The  students  (and  the  two  instructors)  were  each  issued  a  packet  containing  three  items: 

(1)  the  Oddi  Continuing  Learning  Inventory  (OCLI);  (2)  a  demographic  questionnaire,  and,  (3) 
three  blank  sheets  of  paper  and  instructions  with  an  example  for  completing  a  concept  map  (see 
Appendix  C).  They  were  directed  to  spend  three  minutes  completing  the  demographic 
questionnaire,  eight  to  ten  minutes  completing  the  self-directed  learning  inventory,  and  twenty 
minutes  completing  a  concept  map.  Students  were  provided  instructions  and  an  example  for 
developing  concept  maps.  They  were  told  to  complete  a  map  with  a  focal  concept  of  Strategic 
Estimate  and  to  then  write  all  the  concepts  and  sub-concepts  brought  to  mind  when  the  focal 
concept  was  considered.  Further,  they  were  instructed  to  include  in  the  map  some  representation 
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of  how  the  concepts  were  connected  (if  appropriate).  The  packets  were  coded  to  keep  the  three 
items  together  and  the  groups  separate. 

The  independent  variable  of  this  study  was  the  use  of  a  modified  PBL  approach  in  the 
study  groups.  Dependent  variables  included  three  measures  obtained  from  the  concept  maps;  (1) 
number  of  concepts,  (2)  linkages  among  concepts,  and,  (3)  accuracy  of  concepts,  and  four 
measures  of  orientation  toward  self-directed,  continuing  learning.  These  four  measures  were:  (1) 
the  overall  score  on  the  OCLI;  (2)  general  factor;  (3)  ability  to  be  self-regulating,  and,  (4)  avidity 
of  reading. 

Concept  Map  Scoring 

Concept  maps  were  analyzed  by  scoring  (1)  concepts  (the  number  of  core,  major,  and 
subsidiary  concepts  contained  in  concept  maps);  (2)  linkages  (the  quantity  and  complexity  of 
relationships  among  concepts),  and,  (3)  concept  accuracy.  Different  scores  were  given  for  core, 
major,  and  subsidiary  concepts  in  order  to  measure  hierarchial  relationships  among  concepts, 
which  incorporates  the  idea  of  subsumption  of  concepts  under  more  general  ones.  Linkage 
scoring  was  based  on  the  number  of  connections  made  among  concepts,  as  well  as  on  the 
complexity  of  the  connections  (eg.,  a  simple  line  connecting  two  concepts  versus  a  line  with 
arrows  pointing  in  two  directions).  Linkage  scores  were  used  as  measures  of  progressive 
differentiation,  in  which  new  eoncepts  gain  greater  meaning  as  links  are  formed,  and  of  integrative 
reconeiliation  in  which  new  links  among  related  sets  of  concepts  are  developed  by  learners 
(Rafferty  &  Fleschner,  1993,  Novak  &  Gowin,  1984).  Accuracy  of  concepts  was  used  to 
measure  the  extent  to  which  encoding  specificity  was  achieved,  and,  given  the  focus  of  the  study, 
was  the  major  dependent  variable  Concepts  were  considered  accurate  if  they  were  (1)  concepts 
identified  by  course  authors  and  instructors  as  central  to  strategic  estimates,  and/or  (2)  the  focus 
of  cued  retrieval  strategies  employed  in  the  instructional  approach  used  with  the  study  groups. 

Concept  maps  were  scored  independently  by  the  three  investigators  over  three  iterations 
In  the  first  iteration,  maps  were  scored  independently  and  investigators’  scoring  (not  the  maps 
themselves)  were  compared  for  level  of  agreement.  Major  discrepancies  among  scores  required 
all  investigators,  in  the  second  iteration,  to  re-score  the  particular  concept  maps  that  were  in 
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question  Since  concept  accuracy  was  the  study’s  major  dependent  variable,  an  effort  was  made 
to  arrive  at  total  agreement  among  investigators  for  this  variable.  In  the  few  cases  in  which 
accuracy  scores  remained  discrepant  after  two  iterations,  the  investigators  compared  actual 
concept  maps  in  the  third  iteration  in  order  to  arrive  at  complete  agreement.  Discrepancies  in 
accuracy  scores  were  determined  to  be  the  result  of  investigators  overlooking  concepts  or 
miscalculating  scores  rather  than  incorrectly  applying  the  scoring  algorithm.  The  Pearson  product 
moment  correlation  coefficient  was  used  to  determine  the  inter-rater  reliability  for  concept  and 
linkage  scores.  Level  of  agreement  ranged  from  0.90  to  0  95  for  concept  scores  and  from  0  94  to 
0.97  for  linkage  scores.  One  hundred  percent  agreement  was  reached  for  accuracy  scores 
Scoring  of  Oddi  Continuing  Learning  Inventory 

Investigators  followed  scoring  instructions  for  the  Oddi  Continuing  Learning  Inventory, 
adjusting  reversed  scored  items  and  calculating  adjusted  means  for  subjects’  overall  orientations 
to  self-directed,  continuing  learning  (total  instrument),  as  well  as  for  the  other  three  constructs 
(factors)  measured  by  different  items  of  the  instrument:  (1)  a  general  factor  (15  items);  (2)  ability 
to  be  self-regulating  (3  items);  and,  (3)  avidity  of  reading  (4  items)  (Six,  1989).  Cronbach’s  alpha 
was  employed  to  test  reliability  and  was  found  to  be  0.87  for  the  total  instrument,  0.91  for  the 
general  factor,  0.66,  for  the  ability  to  be  self-regulating  factor,  and  0.74  for  the  avidity  of  reading 
factor  The  instrument  and  its  three  factors  were  judged  to  possess  sufficient  reliability  to  conduct 
further  analyses. 

Analysis 

Since  the  two  study  groups  and  the  two  comparison  groups  were  taught  by  the  same 
instructors  using  the  same  content  and  instructional  strategies,  data  from  groups  were  combined 
for  the  purposes  of  analyses.  Chi-square  analysis  was  employed  to  test  for  independence  between 
study  and  comparison  groups  on  the  basis  of  rank,  service  branch,  gender,  specialty  (combat 
arms,  combat  support,  and  combat  service  support),  last  post  (staff  officer,  leadership  position, 
and  other,  such  as  student  or  instructor),  and  area  of  academic  study  (social  sciences  and 
humanities  or  physical  and  biological  sciences).  A  two-tailed  t-test  was  employed  to  test  for 
difference  among  groups  for  years  of  military  service. 
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A  one-tailed  t-test  was  employed  to  test  for  differences  between  groups  for  the  dependent 
variables  of  concepts,  linkages,  accuracy,  and  the  four  measures  of  orientation  toward  self- 
directed,  continuing  learning  (overall  score  and  scores  on  the  instrument’s  three  factors).  A  one- 
tailed  t-test  was  selected  because  previous  theory  and  research  suggest  that  the  instructional 
approach  and  strategies  employed  in  the  study  group  should  result  in  higher  scores  on  dependent 
variables  A  p-level  of  0.05  was  selected  as  the  decision  criterion  to  accept  statistically  significant 
differences. 

Results 

Sixty-four  persons  were  members  of  the  four  staff  groups  that  participated  in  the  study 
Of  the  64  potential  subjects,  60  completed  concept  maps,  61  provided  demographic  information, 
and  55  completed  the  OCLI.  However,  one  person  did  not  answer  all  OCLI  items,  leaving  54 
useable  learning  inventories.  Of  the  sixty-one  subjects  who  provided  demographic  information, 
eight  (13  1%)  were  captains,  48  (78.7%)  were  majors,  and  five  (8.2%)  were  lieutenant  colonels. 
All  captains  had  been  selected  for  promotion  to  the  rank  of  major  and  were  awaiting  their  actual 
promotion,  which  would  occur  within  a  few  months.  Forty-eight  (78.7%)  subjects  were  in  the  U 
S.  Army,  four  (6.6%)  were  in  the  Air  Force,  three  (4.9%)  in  the  Navy,  and  six  (9.8%)  were 
officers  in  other  countries.  Fifty-eight  (95. 1%)  subjects  were  male,  while  three  (4.9%)  were 
female  Their  years  of  military  service  ranged  from  10  to  23  years,  with  the  average  being  13.70 
years  (S.D.  =  2.47). 

Thirty-four  (55.7%)  subjects’  specialty  was  combat  arms,  1 1  (18%)  subjects’  specialty 
was  combat  support,  and  16  (26.2%)  had  a  specialty  of  combat  service  support.  Thirty-eight 
(62.3%)  had  been  in  staff  positions  in  their  previous  assignments,  nine  (14.8%)  had  held 
leadership  positions,  and  14  (23%)  held  other  types  of  positions,  such  as  instructor  or  student. 
Fifty-four  subjects  provided  data  on  the  subject  matter  of  their  academic  degrees,  and  of  this 
number,  36  (66.7%)  had  their  degrees  in  the  social  sciences  or  humanities,  while  18  (33.3%) 
earned  degrees  in  the  physical  or  biological  sciences. 

No  statistically  significant  differences  were  found  in  the  composition  of  study  and 
comparison  groups  on  the  basis  of  these  demographic  variables.  This  finding  provides 
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confirmation  that  group  assignment  procedures  used  by  the  CGSOC  result  in  heterogeneity  within 
staff  groups  and  like-composition  among  different  staff  groups 

One  statistically  significant  difference  was  found  between  study  and  comparison  groups 
Accuracy  scores  of  subjects  in  study  groups  were  significantly  higher  (p  =  0.012)  than  those  of 
subjects  in  comparison  groups.  No  statistically  significant  differences  were  found  for  concept 
scores,  linkage  scores,  or  the  four  measures  of  subjects’  orientation  to  self-directed,  continuing 
learning  The  results  of  these  analyses  are  portrayed  in  Table  1 
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Table  1 


Analysis  of  Dependent  Variables  by  Study  and  Comparison  Groups 


Variable 

Study 

Group 

Mean 

(n) 

Comparison 

Group 

Mean 

(n) 

t 

(df) 

P 

Concepts 

56.59 

49.81 

1  00 

.160 

(29) 

(31) 

(50.1) 

Linkages 

41.34 

35.51 

1,02 

.181 

(29) 

(31) 

(58) 

Accuracy 

4.38 

3.08 

2.32 

.012* 

(29) 

(31) 

(58) 

Orientation  to  Self-Directed 

Continuina  Learning 

Overall  Orientation 

5.09 

5.13 

-.21 

.83 

(27) 

(27) 

(42.7) 

General  Factor 

5.53 

5.65 

-.47 

.64 

(27) 

(27) 

(38  4) 

Ability  to  be  Self- 

Regulating 

4.27 

4.14 

.35 

.73 

(27) 

(27) 

(52) 

Avidity  of  Reading 

4.28 

4.14 

,33 

.74 

(27) 

(27) 

(52) 

Note.  Degrees  of  freedom  for  three  variables  (concepts,  overall  orientation,  and  general  factor) 
differ  from  usual  degrees  of  freedom  since  adjustment  in  t-values  were  required  due  to  unequal 
group  variances. 

*p<.05,  one-tailed  test 
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Discussion  and  Implications 

The  cued  retrieval  strategies  of  the  modified  PEL  approach  were  shown  to  have  a  positive 
impact  on  the  accuracy  of  subjects’  recall  as  measured  by  concept  maps.  This  finding  underscores 
the  efficacy  of  this  modification  for  continuing  military  education.  This  is  particularly  important  if 
accuracy  in  application  of  conceptual  frameworks  is  key  to  competent  leadership  performance. 
Transfer  of  accurate  recall  to  practice  was  beyond  the  scope  of  this  study.  However,  this  study’s 
findings  suggest  that  research  on  this  topic  is  warranted. 

Although  no  significant  differences  were  found  for  the  concepts  and  linkages  variables, 
study  group  subjects  nonetheless  scored  higher  on  these  variables  than  comparison  group 
subjects.  Their  concept  maps  were  more  elaborate,  demonstrating  more  levels  of  concepts 
(hierarchy)  and  more  complexity  in  linkages  among  concepts.  This  directionality  in  results 
suggests  that  a  modified  PEL  approach  may  also  have  potential  for  positively  affecting  the 
elaboration  (restructuring  and  tuning),  not  just  accretion,  of  cognitive  schemata.  More  research 
on  this  topic  is  also  recommended. 

The  lack  of  significant  differences  between  the  study  and  comparison  groups  on  measures 
of  orientation  to  self-directed,  continuing  learning  is  attributable  to  two  factors.  First,  observed 
instructional  strategies  and  techniques  in  the  study  groups  focused  less  than  planned  on  subjects’ 
developing  learning  issues  and  accessing  resources  in  order  to  respond  to  cued  retrieval  questions 
and  to  dilemmas  posed  by  the  notational  practical  exercise.  Second,  while  students  in  medical 
Problem-Eased  Learning  curricula  have  been  shown  to  exhibit  significantly  higher  orientations  to 
self-directed,  continuing  learning  than  students  in  conventional  medial  curricula,  these  differences 
appear  later  in  the  medical  students’  programs  (Shulman,  1995).  The  effect  that  Problem-Eased 
Learning  has  on  the  development  of  self-directed  learning  orientations  appears,  therefore,  to 
require  considerable  time,  a  condition  that  was  not  present  in  this  study.  (The  course  in  which  the 
modified  PEL  approach  was  implemented  lasted  only  two  weeks.)  The  impact  that  any  PEL 
strategy  (and  its  interaction  with  time)  has  on  the  development  of  self-directed  learning 
orientations  requires  further  study. 
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APPENDIX  A 


U.S.  AilMY  COMMAND  AND  GENERAL  STAFF  COLLEGE  C5 10-7 

STRATEGIC,  OPERATIONAL  AND  JOINT  ENVIRONMENTS 
Lesson  7.  Strategic  Estimate 

'  Appendix  4  to  the  Advanced  Sheet,  Lesson  7.  Practical  Exercise  Scenario  and  Student  Requirement: 
Strategic  Estimate — Korean  Peninsula/Northeast  Asia 

•  *  *  THIS  PRACTICAL  EXERCISE  SCENARIO  (NOTIONAL)  IS  FOR  USACGSC  USE  ONLY  *  *  * 


1.  BACKGROUND. 

a.  Kim  Jong  D,  appears  to  have  taken  over  the  reins  of  power  after  the  death  of  his  father,  Kim  Q  Sung,  in  July 
of  1994.  However,  Kim  Jong  U  has  not  appeared  in  public,  except  on  rare  important  state  occasions,  and  rumors 
continue  to  circulate  on  his  fitness  and  competence  as  the  leader  of  the  Democratic  People's  Republic  of  Korea 
PPRK).  Economic  problems  continue  to  plague  the  regime,  but  the  most  significant  problems  are  related  to  the 
US-DPRK  Framework  Agreement. 

b.  In  October  1994,  the  US  and  the  DPRK  signed  an  agreement  (actually  three  separate  documents)  to  resolve 
the  complicated  dispute  over  North  Korea's  nuclear  weapons  program.  In  the  winter  of  1994-95  the  US  supplied 
oil  to  the  North.  Implementing  the  accords,  the  DPRK  shut  down  its  reactor  at  Yongbyon  and  allowed  a  limited 
inspection  regime  by  the  International  Atomic  Energy  Agency  (IAEA),  a  United  Nations  (UN)  agency.  The  DPRK 
continues  to  search  for  a  means  to  raise  hard  currency  to  feed  its  people  and  to  maintain  its  huge  defense 
establishment.  Neither  China  nor  Russia  appear  willing  to  provide  anything  but  lunited  barter  or  cash  and  carry 
trade  with  the  North.  Weapons  sales  by  the  North,  especially  to  Iran  and  Iraq,  while  earning  needed  fimds,  continue 
to  be  widely  criticized. 

c.  In  South  Korea  and  the  West,  many  see  tire  election  of  the  reform-minded  Kim  Young  Sam,  the  first 
non-military  President  of  the  Republic  of  Korea  since  Syngman  Rhee  in  the  1950s,  as  a  key  event  in  the 
democratization  program  and  a  signal  of  even  stronger  econorruc  growth  and  maturity  in  the  South.  However,  the 
increasingly  open  political  enviromnent  has  also  led  to  demands  for  more  autonomy  and  independence  on 
international  issues  and  greater  leadership  by  the  ROK  in  its  own  security  affairs. 

d.  There  is  significant  domestic  debate  in  the  ROK  and  in  the  U.S.  Congress  concerning  the  U.S.  response  to 
the  DPRK's  alleged  nuclear  arms  development  Many  question  the  appropriateness  of  the  U.S.  diplomatic  and 
military  efforts  to  date.  Some  critics  call  for  complete  disclosure  and  an  immediate  verified  end  to  any  nuclear 
weapons  program  in  the  North. 

e.  Increased  direct  talW<8  and  dialogue  between  the  two  Koreas  had  been  steadily  improving  after  signing  a 
non-aggression  pact  in  1992,  agreeing  to  nuclear  inspections,  accepting  individual  seats  in  the  United  Nations,  and 
resolving  the  DPRK's  threat  to  pull  out  of  the  Nuclear  Non-proliferation  Treaty  in  June  1993.  A  renewed  effort 
following  the  Carter  visit  led  to  the  October  Framework  Agreements.  However,  some  analysts  asserted  that  the 
North  does  not  intend  to  allow  joint  inspections  of  all  of  its  nuclear  sites  and  could  relocate  its  facilities  to 
well-disguised,  remote,  underground  facilities. 

2.  SITUATION. 

a.  The  current  rrri«ri.!  began  early  1995  with  tiie  Nortii  Korean  refusal  to  accept  the  two  light  water  reactors  from 
South  Korea  as  originally  agreed  to  in  the  October  1994  accords.  After  continued  failed  talks  through  the  Summer 
of  1995,  the  DPRK  delegation  walked  out  and  returned  to  North  Korea  with  no  indication  of  a  time  or  place  of 
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renewed  talks.  While  there  has  been  no  official  abrogation  of  the  October  1994  agreement,  IAEA  inspectors  were 
asked  to  leave.  The  Clinton  Administration  has  warned  that  it  will  impose  economic  sanctions  imless  the  DPRK 
returns  to  the  talks. 


b.  On  6  July  1995,  the  Korean  People’s  Army  (KPA)  tested  their  No  Dong  missile  with  a  launch  off  the  east 
coast  into  the  Sea  of  Japan.  The  Japanese  government  protested  this  unprecedented  action  and  Japanese  press 
portrayed  this  as  a  deliberate  warning  to  Japan  not  to  support  UN  and  U.S.  economic  sanctions  against  the  DPRK. 
The  same  day  there  were  reports  in  the  Middle  East  of  additional  North  Korean  weapons  shipments  to  Iran  and  Iraq 
through  a  third  party.  The  DPRK  warned  the  U.S.  about " interruptions  of  international  shipping”  during  its  press 
announcement  warning  against  economic  sanctions.  Four  days  ago  the  ROK  army  reported  small  arms  fire  west 
of  Panmimjom.  The  details  of  this  activity  are  not  available.  However,  since  this  incident,  a  heated  exchange  of 
escalating  accusations  has  caused  the  political  and  the  military  situation  to  deteriorate  and  both  countries'  militaries 
are  m  a  heightened  state  of  alert.  There  are  additional  reports  of  small  arms  fire  across  the  DMZ  and  both  sides 
allege  troop  movement  and  exercise  activity  by  the  other  in  the  vicinity  of  the  DMZ.  The  ROK  govermnent  has 
requested  additional  U.S,  intelligence  assets  to  monitor  the  situation,  and  that  U.S.  Army  and  Air  Force  personnel 
in  Korea  for  an  armual  CPX  (Ulchi  Focus  Lens)  remain  in-country. 

d.  Yesterday,  Republic  of  Korea  President  Kim  expressed  grave  concern  over  the  North’s  purpose  for 
precipitating  an  incident  at  this  time  and  its  ultimate  objective.  This  may  be  the  first  major  test  for  the  DPRK’s  new 
leader,  Kim  Jong  fl.  There  are  rumors  that  senior  DPRK  military  leaders,  who  at  first  were  supportive  of  the  new 
lea^r,  are  dissatisfied  with  the  pace  and  direction  of  the  changes  in  North  Korea, 

3.  REQUIREMENT.  The  CJCS  must  attend  a  NSC  meeting  late  tomorrow  to  discuss  the  reemerging  “crisis"  in 
Korea.  He  requested  an  immediate  in-house  strategic  assessment  of  the  situation  on  the  Korean  Peninsula  and 
scheduled  a  video  conference  with  USCINCPAC  in  preparanon  for  this  meeting.  To  prepare  the  CINC,  USP  ACOM 
J-50  is  chairing  an  ad  hoc  working  group  to  develop  courses  of  actions  and  recommendations  for  the  CINC  to  use 
in  his  discussions  with  CJCS.  As  members  of  the  USPACOM  crisis  planning  cell  (PACOM  staff  officers  and 
selected  representatives),  analyze  this  situation,  develop  courses  of  action  and  make  a  recommendation.  You'll  have 
less  than  an  hour  with  the  CINC.  As  you  know,  he  understands  the  region  well  and  likes  to  understand  the  logic 
of  your  analysis,  so  you'll  have  to  be  concise.  He'll  want  to  understand  who  are  the  key  players  and  what  are  their 
interests  and  objectives.  He’ll  also  be  concerned  about  the  synchronization  of  all  of  the  instruments  of  national 
power. 

4.  ADMINISTRATIVE  NOTES: 


a.  Your  DJCO  instructor  will  assign  the  following  staff  positions  and  provide  the  appropriate  resource  packets: 


J-50 

J-511 

J-511 

J-511 

J-511 

J^l 

FPAl 

Liaison 

Liaison 

Representative 

Representative 


Deputy  Director,  Strategic  Plans  and  Policy  Directorate 

Country  Desk  Officer — Korea 

Country  Desk  Officer — Japan 

Country  Desk  Officer — Russia 

Country  Desk  Officer — China 

Chief,  International  Logistics  Division 

Assistant  Foreign  Policy  Advisor 

COMUSKOREA 

COMUSJAPAN 

CDR,  MARFORPAC 

CINCPACFLT 
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APPENDIX  B 


Implementing  Two  Elements  (Activation  of  Prior  Knowledge  &  Identification  of  Learning 
Issues)  of  Problem-Based  Learning  in  C510  Lesson  7:  Strategic  Estimate  -  Korean 
Peninsula/Northeast  Asia 


Format  for  PE  Scenario  of  Lesson  7: 

•  In  2  Groups,  Students  read  “Background”  Section  of  Scenario  and  respond  to 
Questions 

•  Briefing  on  Background:  Regroup  in  Staff  group  and  discuss  responses  to 
questions 

•  In  2  Groups,  Students  read  “Situation”  Section  of  Scenario  and  respond  to 
Questions 

•  Briefing  on  Situation:  Regroup  in  Staff  group  and  discuss  responses  to  questions 
In  Staff  Group,  Discuss  COA  and  Strategic  Estimate 

(Learning  Issues  will  be  identified  in  conjunction  with  this  discussion) 

•  2  Students  (one  for  each  group)  will  be  given  the  Questions  the  day  prior  to  the  exercise 
Students  will  facilitate  group’s  discussion  of  the  questions 

I.  “Background”  Questions  [The  Following  questions  are  for  groups  of  7-9  students) 
[Questions  to  be  asked  after  students  have  read  the  Background  Section  of  the  PE  Scenario] 

1 .  What  instruments  of  power  have  been  employed  to  date?  (Probe  for  all) 

[DIME/Lesson  Ij 

2  What  are  categories  of  U  S  national  interest  that  might  merit  the  use  of  military  power  in 
Korea?  [National  Interest/Lesson  0  &  1] 

3  What  are  the  sources  of  power  to  this  point?  [Sources  of  Power/Lesson  I  ] 

4.  What  are  the  joint  and  combined  structures  in  Northeast  Asia?  How  do  they  function  and  what 
are  combined/multinational  considerations  associated  with  them?  [Lesson  3  &  6] 

5.  What  are  capabilities  and  limitations  of  U  S.  services  in  the  region?  [Lesson  4] 

6.  Which  view  —  Kaplan,  Tofler,  Huntington  —  best  explains  today’s  environment  in  Korea‘S 
[Lesson  2] 

7  To  summarize  to  this  point.  What’s  Known'’  What’s  Unclear?  What’s  Presumed? 
[Identification  of  Learning  Issues] 
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II.  “Situation”  Questions  (The  following  questions  are  for  groups  of  7-9  students) 
[Questions  to  be  asked  after  students  have  read  the  Situation  Section  of  the  PE  Scenario] 

la  What  recognition  is  given  to  Northeast  Asia  in  the  NMS  and  the  NSS?  What  are  the  three 
goals  of  NSS  for  the  region? 

lb.  What  recognition  is  given  to  Korea  in  the  NMS  and  the  NSS?  What  are  the  three  goals  of 
NSS  for  Korea?  As  described  in  this  situation,  which  strategic  goal(s)  is/are  most  at  risk? 
[Lesson  2  and  Advanced  Readings  for  Lesson  7 j 

2  Who  are  the  actors  in  this  situation? 

3  Given  this  situation: 

a.  What  instruments  of  power  should  be  employed? 

b  What  categories  of  U  S.  national  interest  would  merit  the  use  of  military  power? 
c.  What  are  the  sources  of  power'll 

4  What  are  the  similarities  between  this  situation  and  the  other  PEs  we’ve  had  on  Korea‘S 

5  What  are  the  differences  between  this  situation  and  the  other  PEs  we’ve  had  on  Korea‘S 
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APPENDIX  C 


Demographic  Information 

Note:  All  information  gathered  Is  for  researchers’  use  only  to  provide 
context  for  the  analysis  of  your  conceptual  map.  Your  response  is 
completely  voluntary  and  will  be  held  in  the  most  strict  confidence. 

1)  Rank  Captain,  Major,  LTC  (or  equivalent) 

2)  USAF  USN  Army  Marine  International  (Country  _ ) 

3)  Male  Female 

4)  Specialty(ies)  (E.g.,  Infantry) _ _ 

5)  Years  of  service _ . 

6)  Last  duty  position  (not  location)  _ 

7)  Favorite  hobby  _ 

8)  Academic  Area(s)  In  which  degree(s)  was(were)  awarded: 


Making  a  Concept  Map 


A  concept  map  is  a  graphical  representation  of  concepts  and  their 
interrelationships.  In  a  concept  map.  related  concepts  are  represented  as  “nodes"  and 
the  specific  relationship  between  two  concepts  is  indicated  by  “linking  words”.  Your 
task  is  to  draw  a  map  that  centers  around  the  concept  of  the  “strategic  estimate”,  but 
first  I  want  to  show  you  an  example  of  a  concept  map  I  developed  for  the  “wearing  of 
military  uniforms.” 

Here’s  the  way  I  made  a  concept  map  for  the  wearing  of  military  uniforms.  To 
make  the  map,  I  started  by  thinking,  for  a  few  minutes,  about  the  focal  concept,  i.e.,  the 
wearing  of  military  uniforms.  I  wrote  down  the  key  concepts,  terms,  and  examples  that 
I  thought  were  important  to  describe  the  wearing  of  military  uniforms.  I  placed  them  in 
a  single  column  as  shown  in  the  example. 

When  the  list  was  complete,  I  sorted  the  concepts,  terms  and  examples  into 
clusters  according  to  the  extent  to  which  they  were  related  by  making  larger  categories 
in  Column  2.  I  then  linked  the  concepts  from  Column  1  to  the  major  categories  in 
Column  2  with  lines.  Your  groupings  and  examples  regarding  the  strategic  estimate 
will  reflect  your  personal  judgments  about  the  closeness  of  associations  among  the 
terms.  Each  person’s  concept  map  will  be  different,  so  don’t  worry  about  whether  your 
map  is  the  same  as  someone  else’s,  or  if  they  have  more  terms  or  groups  than  you  do. 

Next,  I  constructed  a  map  and  arranged  and  linked  the  focal  concept,  the 
wearing  of  military  uniforms,  and  the  clusters  of  concepts  with  lines.  I  positioned  the 
focal  concept  on  the  paper  with  plenty  of  room  to  arrange  my  clusters  around  the 
concept.  I  used  arrows  to  indicate  the  direction  of  the  relationship  that  existed 
between  the  concepts;  relationships  may  be  in  one  direction  or  both  directions.  You 
should  be  sure  to  include  relationships,  or  cross-links,  that  exist  between  different 
clusters  of  concepts  as  well  as  relationships  within  the  clusters.  If  you  must  draw  one 
connecting  link  so  that  it  intersects  another  connecting  link,  but  you  do  not  wish  to 
indicate  the  corresponding  concepts  are  related,  draw  a  small  circle  around  the  point 
of  intersection. 

Finally,  I  labeled  the  connecting  lines  on  my  map  to  show  valid  and  meaningful 
relationships  between  the  concepts.  You  should  work  with  one  pair  of  concepts  at  a 
time.  Some  examples  of  linking  words  are;  is  a,  is  part,  has  property,  consists  of, 
involves,  with,  by,  and  requires. 

Now  you  will  do  the  same  mapping  process  for  the  strategic  estimate. 
Remember,  each  person’s  concept  map  will  be  different.  Your  map  is  one  indication  of 
how  you  think  about  the  strategic  estimate,  and  there  is  no  right  nor  wrong  way  to 
arrange  the  terms.  This  is  not  a  test  on  the  strategic  estimate! 
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Column  1 


Column  2 


Service  cap 
Tradition 
Symbol 
Ironing 

Official  Photos 
Raincoat 
Low  Quarters 
Boots 
Shirts 
Medals 
Class  A 
Class  B 
BDUs 
Unit  Patch 
Combat  Patch 
Rank 

Branch  Insignia 

Haircut 

Polish 

Fitting 

Occasions 

Costs 

Mess  Blues 

Whites 

Dress  Blues 

Ties 

Socks 

Boots 

Beits 

Headgear 

Caps 

Garison  Cap 
Gortex  Parka 
Camouflage 
Precedence 


Accessories 


Add'Ons 


Appearance 


Occasions 
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